System and method for generating a 2d image using mammography and/or tomosynthesis image data

ABSTRACT

The invention includes a method including the steps of obtaining a plurality of images, each of the images in the plurality having at least one corresponding region, generating a merged image, the merged image also having the corresponding region. The step of generating includes selecting an image source from the plurality of images to source image data for the corresponding region in the merged image by comparing attributes of the corresponding regions of the plurality of images to identify the image source having preferred attributes.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/220,913, filed Apr. 1, 2021, which is a continuation of U.S. patentapplication Ser. No. 16/792,182, now U.S. Pat. No. 10,978,026, filedFeb. 14, 2020, which is a continuation of U.S. patent application Ser.No. 16/013,701, now U.S. Pat. No. 10,573,276, filed Jun. 20, 2018, whichis a continuation of U.S. patent application Ser. No. 14/360,389, nowU.S. Pat. No. 10,008,184, filed May 23, 2014, which is a National Phaseentry under 35 U.S.C § 371 of International Patent Application No.PCT/US2012/066526, having an international filing date of Nov. 26, 2012,which claims priority under 35 U.S.C. § 119 to U.S. Provisional PatentApplication No. 61/563,785, filed Nov. 27, 2011. This application isalso related to U.S. patent application Ser. No. 12/471,981, now U.S.Pat. No. 8,571,289, filed May 26, 2009; U.S. patent application Ser. No.12/276,006, now U.S. Pat. No. 7,760,924, filed Nov. 21, 2008; U.S.patent application Ser. No. 11/827,909, now U.S. Pat. No. 7,616,801,filed Jul. 13, 2007; U.S. patent application Ser. No. 11/604,069, nowabandoned, filed on Nov. 24, 2006; and U.S. patent application Ser. No.11/271,050, now U.S. Pat. No. 7,577,282, filed Nov. 11, 2005. Each ofthe above applications is hereby incorporated by reference.

FIELD

This patent specification pertains to x-ray mammography andtomosynthesis, and more specifically to techniques and equipment foracquiring and/or synthesizing, processing, storing and displayingmammograms, tomosynthesis projection images, synthesized two-dimensional(2D) images and/or tomosynthesis reconstructed images, and to medicalimage softcopy reading systems, to hanging protocols and to othermedical image display features.

BACKGROUND

Mammography has long been used to screen for breast cancer and otherabnormalities and for diagnostics. Traditionally, mammograms were formedon X-ray film, but more recently flat panel digital imagers have beenintroduced that acquire a mammogram in digital form and therebyfacilitate analysis and storage and provide other benefits as well.Further, substantial attention and technological development has beendedicated towards obtaining a three-dimensional image of the breast,using methods such as breast tomosynthesis. In contrast to the 2D imagesgenerated by legacy mammography systems, breast tomosynthesis systemsconstruct a 3D image volume from a series of 2D projection images, eachprojection image obtained at a different angular displacement of thex-ray source relative to the image detector as the x-ray source isscanned over the detector. The constructed 3D image volume is typicallypresented as a plurality of slabs or slices of image data, the slabsgeometrically reconstructed on planes parallel to the imaging detector.The reconstructed tomosynthesis slices reduce or eliminate the problemscaused by tissue overlap and structure noise in single slicetwo-dimensional mammography imaging by permitting a radiologist toscroll through the slabs and view underlying structures.

Tomosynthesis systems have recently approved for breast cancer screeningand diagnosis. The assignee of this patent specification, Hologic, Inc.,has demonstrated at trade shows in this country a fused, multimodemammography/tomosynthesis system that takes either or both types ofmammogram and tomosynthesis images, either while the breast remainsimmobilized or in different compressions of the breast. Other companieshave proposed the introduction of systems which are dedicated totomosynthesis imaging, i.e., which do not include the ability to alsoacquire a mammogram.

Restricting systems to tomosynthesis acquisition and image display maypresent an obstacle to acceptance of the tomosynthesis imagingtechnology, as medical professionals have grown accustomed to screeningand analysis of mammogram images. Mammograms offer good visualization ofmicro-calcifications, and can offer higher spatial resolution whencompared with tomosynthesis images. While tomosynthesis images providedby dedicated breast tomosynthesis systems in the art have otherdesirable characteristics (i.e., better visualization of structures),such systems do not leverage the existing interpretation expertise ofmedical professionals.

One method of leveraging existing medical expertise to facilitate thetransition to tomosynthesis technology was described in U.S. Pat. No.7,760,924, entitled “System and Method for Generating a 2D Image from aTomosynthesis Data Set.” The '924 patent describes a method ofgenerating a synthesized 2D image which may be displayed along withtomosynthesis projection or reconstructed images, to assist in screeningand diagnosis.

SUMMARY

According to one aspect of the invention, it is realized that animproved synthesized 2D image may be obtained by merging the mostrelevant data from a plurality of data sources. In one embodiment, themerging is performed using a combination of 2D and 3D image data,wherein the 2D image may include either an acquired mammogram, asynthesized mammogram, or a tomosynthesis projection image, and the 3Dimage data may comprises a reconstructed 3D data set. In an alternateembodiment, the merged data is formed using 3D projection images and/orreconstruction images. The improved synthesized image, referred toherein as a ‘merged’ image I_(MERGE), incorporates the most relevantinformation from all acquired and computer generated data sets into one‘supreme’ 2D image for display on a workstation. Thus, regions of pixelsin the displayed merged image may be sourced by different images in adata set including but not limited to one or more of an acquired 2Dimage (mammogram or tomosynthesis projection image), a synthesized 2Dimage, or a reconstructed tomosynthesis slice or slab. The particularregions may be identified statically (i.e., within a particular grid),or dynamically, and may range in granularity from one pixel to allpixels in the image. With such an arrangement, the radiologist mayquickly view a large number of regions of interest within a breast whilereferencing only a single 2D image, thereby increasing the performanceand efficiency of breast cancer screening and diagnosis.

According to a further aspect of the invention, a map is automaticallygenerated for each region in the merged image, identifying theparticular image that sourced the region in the merged image. Aninterface feature may be provided that enables the origin source imageto be displayed when the region is selected by the user. The originsource image may be displayed in a variety of manners, e.g., overlaidover the merged image to allow toggling between the two images ordisplay using cine mode, or displayed adjacent to the merged image, etc.

According to a further aspect of the invention, it is determined thatparticular types of images may include different types of relevantinformation. For example, calcifications are best visualized in 2Dmammograms, while masses are best visualized using 3D reconstructedimages. In one embodiment, different filters are applied to each of thedifferent types of images (i.e., 2D and 3D), where the filters areselected to highlight the particular characteristics of the images whichare best displayed in the respective imaging mode. Appropriate filteringof the respective types of images prior to the merge ensures that thefinal merged image includes the most relevant information that can beobtained from all image types. Alternatively, the type of filtering thatis performed for the various images may be defined via user input, whichpermits a user to select a ‘merge mode’, for example, geared towardshighlighting masses, calcifications, or making the merged image appearto be a particular image type, such as a 3D reconstructed slice, or a 2Dmammogram.

Merging of the images may be done in a variety of ways. According to oneembodiment, general purpose Computer Assisted Diagnosis (CAD) algorithmsare used to identify features within each of the 2D and 3D images. Thevarious features are assigned weights, and the merged image is built byselecting the image having the region with the most significant weight.The size of the region may vary in granularity from one pixel to many(or even all) pixels, and may be statically pre-defined, or may havemargins which vary in accordance with the varying thresholds of thesource images.

In accordance with a further aspect of the invention, it is envisionedthat the merged image may be pre-processed and stored followingtomosynthesis acquisition, or dynamically generated in response to arequest for a merged image at a radiologist work station.

It is realized that the visualization of such a merged image may havesome drawbacks. For example, there may be neighboring regions in themerged image which exhibit bright calcifications but in fact are sourcedfrom image slices which are distant from one another in the z-plane.Therefore, what may appear to as a mass of calcifications in the mergedimage may, in fact, be calcifications which are distributed throughoutthe breast and thus do not actually represent a calc cluster thatrequires further review. According to a further aspect of the invention,this problem is overcome by the inclusion of a cluster spread indicator.The cluster spread indicator is a graphical indicator provided with themerged image, and visually indicates the distribution of calcificationsalong the z-plane, allowing the medical professional to quickly assesswhether a group of calcifications comprise a calcification cluster.

One aspect of the invention includes a method including the steps ofobtaining a plurality of images, each of the images in the pluralityhaving at least one corresponding region, generating a merged image, themerged image also having the corresponding region, the step ofgenerating including: selecting an image source from the plurality ofimages to source image data for the corresponding region in the mergedimage by comparing attributes of the corresponding regions of theplurality of images to identify the image source having preferredattributes.

The foregoing aspect can include any one or more of the followingembodiments. The preferred attributes includes attributes indicative ofregions of interest, such as cancers, or alternatively such as moreaccurate representation of breast density or breast anatomy (e.g.truthful breast-border/nipple appearance). In general, any attributecapable of delivering a high/better-quality image can be relevant here.The method can further include filtering the plurality of images tomagnify attributes of the plurality of images. The filtering can applydifferent filters to images of different types. The method can furtherinclude visually indicating a distribution of the preferred attributesin the corresponding region by providing a histogram to illustrate adepth of the preferred attributes. The plurality of images can comprisea 2D image and a 3D image. The plurality of images can be selected froma group consisting of tomosynthesis projection images, reconstructedtomosynthesis slices, mammograms, and synthesized two dimensionalimages. The method can further include generating a map for a region inthe merged image, whereby the map identifies at least one of theplurality of images that sourced the region in the merged image. Themethod can further include displaying the at least one of the pluralityof images that sourced the region when the region is selected by a user.The at least one of the plurality of images that sourced the region canoverlay the merged image such that a user can view both types of imagesat the same time or by toggling between the types.

According to another aspect of the invention, a display workstationcomprises an interface, the interface including a mechanism which, uponselection, results in the generation and/or display of a merged image,the merged image comprising a two dimensional synthesized imagecomprising a plurality of regions, wherein at least two of the regionsare sourced by different images selected from a group of images, andwherein the group of images include tomosynthesis projection images,tomosynthesis reconstruction slices, mammograms and synthesized 2Dimages.

The foregoing aspect can include any one or more of the followingembodiments. The interface can further include a mode selectionmechanism enabling a user to select a mode of either generation ordisplay of the merged image. The interface can further include amechanism for selecting a region of the merged image and a mechanism fordisplaying three-dimensional information related to the selected regionon a display station. Additionally or alternatively, the generation ofthe merged display does not take place at the workstation, but canhappen in another part of the system. The interface can display a mapfor at least one of the plurality of regions in the merged image suchthat a user can select at least one of the plurality of regions todisplay one of the group of images that sourced the at least one of theplurality of regions. The interface can allow a user to view both of ortoggle between the merged image and the one of the group of images thatsourced the at least one of the plurality of regions. The interface canprovide a graphical indicator along with a display of the merged imagesuch that the graphical indicator illustrates a depth of attributeswithin the regions. The interface can display simultaneously,sequentially, or in toggle mode two or more of the merged image, thetomosynthesis projection image, the tomosynthesis reconstruction slice,the mammogram and the synthesized 2D image.

The present invention can include any one or more of the foregoingaspects and/or embodiments, in any combination, and can include any oneor more of any of the details described herein.

These and other aspects of the invention will be described in moredetail with regard to the below figures.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating flow of data through a systemwhich includes a combination mammography/tomosynthesis acquisitionstation or a tomosynthesis only acquisition station to acquiretomosynthesis and/or mammography images, and includes image mergetechnology of the present invention for providing a supreme synthesizedtwo dimensional image which merges the most relevant data from allavailable data sources into one 2D image;

FIG. 2 is diagram illustrating the data flow of a series oftomosynthesis slices, a synthesized 2D mammogram through the image mergetechnology of the present invention to generate at least one of a mergedimage I_(MERGE) and a merge map;

FIG. 3 is a diagram illustrating an alternate form of merged image,wherein region boundaries are dynamically identified during merge imagebuild;

FIG. 4 is flow diagram illustrating exemplary steps that may beperformed during an image merge process of the present invention;

FIGS. 5A and 5B illustrate a display of a merged image, and a resultantdisplay of a source image when a region is selected by a user; and

FIG. 6 is a diagram of an exemplary merged image display, whichincorporates an optional cluster spread indicator of the presentinvention.

FIG. 7 is an x-ray image acquisition, processing and display system.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In describing preferred embodiments illustrated in the drawings,specific terminology is employed for the sake of clarity. However, thedisclosure of this patent specification is not intended to be limited tothe specific terminology so selected and it is to be understood thateach specific element includes all technical equivalents that operate ina similar manner.

The following abbreviations shall have the following definitionsthroughout this application. The notation Mp refers to a conventionalmammogram, which is a two-dimensional projection image of a breast andencompasses both a digital image as acquired by a flat panel detector oranother imaging device and the image after conventional processing toprepare it for display to a health professional or for storage, e.g. inthe PACS system of a hospital or another institution.

Tp refers to an image that is similarly two-dimensional but is taken ata respective tomosynthesis angle between the breast and the origin ofthe imaging X-rays (typically the focal spot of an X-ray tube), and alsoencompasses the image as acquired as well as the image after beingprocessed for display or for some other use. Tr refers to an image thatis reconstructed from images Tp, for example in the manner described insaid earlier-filed patent applications, and represents a slice of thebreast as it would appear in a projection X-ray image of that slice atany desired angle, not only at an angle used for Tp or Mp images.

The term Ms refers to synthesized 2D projection images which simulatemammography images, such as a craniocaudal (CC) or mediolateral oblique(MLO) images, and are constructed using tomosynthesis projection imagesTp, tomosynthesis reconstructed images Tr or a combination thereof. Msimages may be provided for display to a health professional or forstorage in the PACS system of a hospital or another institution. Anexample of methods that may be used to generate synthesized 2Dprojection images are described in U.S. patent application Ser. No.12/471,981, filed May 26, 2009, as well as U.S. Pat. No. 7,760,924,filed Nov. 21, 2008, both incorporated herein by reference in theirentireties.

The term I_(MERGE) refers to a 2D image generated by merging togetherany two or more of Mp, Ms, Tp or Tr images.

The terms I_(MERGE), Tp, Tr, Ms and Mp also encompasses information, inwhatever form, that is sufficient to describe such an image for display,further processing, or storage. The images I_(MERGE), Mp, Ms, Tp and Trtypically are in digital form before being displayed, and are defined byinformation identifying properties of each pixel in a two-dimensionalarray of pixels. The pixel values typically relate to respectivemeasured or estimated or computed responses to X-rays of correspondingvolumes in the breast (voxels or columns of tissue). In a preferredembodiment, the geometry of the tomosynthesis images (Tr and Tp),mammography images (Ms, Mp) and the merged image I_(MERGE) are matchedto a common coordinate system as described in U.S. patent applicationSer. No. 11/667,650 “Matching Geometry Generation and Display ofMammograms and Tomosynthesis Images”, filed Nov. 15, 2005 andincorporated herein by reference.

FIG. 1 illustrates flow of data in one example of an image generationand display system which may incorporate the merged image generation anddisplay features of the present invention. It should be understood thatFIG. 1 illustrates a particular embodiment of a flow diagram, withcertain processes happening in a particular serial order or in parallel,the present invention is not generally limited to the performance ofimage processing in any particular order.

FIG. 1 illustrates an image data acquisition system 1 that acquirestomosynthesis image data for Tp images of patients' breasts using thethree dimensional and/or tomosynthesis acquisition methods of any of thesystems available in the art. If the acquisition system is a combosystem, Mp images may also be generated. Some dedicated tomosynthesissystems or combo systems may be adapted to accept and store legacymammogram images (meaning pre-acquired mammogram systems indicated viadashed line and legend Mp_(legacy) in FIG. 1 ) in a Picture Archivingand Communication System (PACS) storage device 2, although it is not arequirement that any Mp images be acquired by the acquisition system orpre-stored.

Following tomosynthesis image acquisition, the projection images Tp aresent to storage device 2, which is preferably a DICOM-compliant PACS.When images are needed for display 5, the Tp images are sent (fromeither acquisition system 1 or from storage device 2) to a computersystem 3 configured as a reconstruction engine that reconstructs the Tpimages into reconstructed image slabs Tr representing breast slices ofselected thickness and at selected orientations, as disclosed in saidearlier-filed patent applications and detailed below. The computersystem may be further configured with 2D synthesis functionality 4,which may operate substantially in parallel with reconstruction engine 3to generate a synthesized 2D image (interchangeably referenced as T2d orMs). The reconstructed slice images Tr are then sent to a display system5 so that they can be viewed. Additionally or alternatively the Trslices can be returned to the storage device. If the reconstructionengine 3 is connected to display 5 via a fast link, then large datasetscan be transmitted quickly. Other images, such as the Ms, Mp and/or Tpimages may also be forwarded to the display unit for concurrent ortoggled viewing.

As shown in FIG. 1 , the imaging and display system of the presentinvention includes a 2D synthesizer for generating 2D images simulatingmammograms taken at both a CC and MLO orientation using a combination ofone or more Tp and/or Tr images. The synthesized 2D images may begenerated dynamically prior to display, as shown in FIG. 1 , and/or maybe stored in storage system 2 for use by other processes.

As will be described in more detail later herein, a set of mode filters7 a, 7 b are disposed between image acquisition and image display. Eachof the filters 7 a and 7 b may additionally include customized filtersfor each type of image (i.e., Tp, Mp, Tr) arranged to highlight certainaspects of the particular types of images. Thus each mode can betuned/configured in a optimal way for a specific purpose. The tuning orconfiguration may be automatic, based on the type of the image, or maybe defined by manual input, for example through a user interface coupledto a display. For example, filters could be provided to define amass/calc-emphasis mode, 3D-tomo-slice-look mode, 2D-mammo-look mode,etc.

According to one aspect of the invention, an image merge processor 6merges relevant image data obtained from a set of available images toprovide a merged image I_(MERGE) for display. The set of availableimages includes at least filtered and/or unfiltered Ms, Mp, Tr and/or Tpimages. It should be noted that although FIG. 1 illustrates that alltypes of images are input into the image merge processor 6 it isenvisioned that the form of merged image shall be manually configurable.Thus a user interface, comprised of a keypad, touch pad, mouse, pulldown menu, button, switch, etc., may be configured to allow a user toselect a particular group of two or more images or image types formerging to produce a supreme 2D image for display. For example, aradiologist may wish to merge two or more reconstructed tomosynthesisslabs to provide a merged image showing the most discernable structuresin a single image. Alternatively, the radiologist may combine the 2Dmammogram image, Mp or Ms, with 3D projection or reconstructed images toobtain a merged image that highlights both calcifications andstructures. Filters applied to each type of image further highlight thetypes of structures or features which are generally most prevalent ordiscernable in the image type; thus a certain filter may be applied tomammography images to highlight calcifications, while a different filtermay be applied to tomosynthesis slices to highlight masses. Filters mayalso be provided to give the particular image a desired look and feel;i.e., make the merged image appear more like a tomosynthesis image, or amammography image.

The display 5 may be the display of an acquisition workstation, or atechnologists review station, or a display that is physically remotefrom the acquisition system or storage device, ie., connected via thenetwork.

A display of the system preferably should be able to display I_(MERGE),Ms, Mp and Tr (and/or Tp) images concurrently (either in separatewindows on the display, on separate monitors of a technologyworkstation, or overlaid) or sequentially or in toggled mode, whereinthe I_(MERGE), Ms, Mp, Tp and Tr images may be those currently acquired,or those that were acquired in previous studies. Thus, in general, thedisplay can simultaneously or sequentially or in toggled mode displaymerged images I_(MERGE), mammograms (Ms, Mp) and tomosynthesis images Tr(and/or Tp) from the current and previous studies. Tr slices can bereconstructed all to the same size, which can be the same as the size ofan Mp or Ms image of the breast, or they can be initially reconstructedto sizes determined by the fan shape of the x-ray beam used in theacquisition and later converted to that same size by appropriateinterpolate]on/extrapolation.

Images of different types and from different sources can be displayed indesirable size and resolution. For example, an image can be displayed in(1) Fit To View Port mode, in which the size of the displayed image sizeis maximized such that the entire imaged breast tissue is visible, (2)True Size mode, in which a display pixel on the screen corresponds to apixel of the image, or (3) Right Size mode, in which the size of adisplayed image is adjusted so that it matches that of another imagethat is concurrently displayed or with which the displayed image is orcan be toggled. For example, if two images of the same breast are takenand are not the same size or do not have the same special resolution,provisions are made to selectively zoom in or zoom out one of them, orzoom both, such that they appear to be the same size on the screen whenthey are concurrently displayed or the user toggles between them, tofacilitate comparison or to otherwise facilitate detection/diagnosis.Known interpolation/extrapolation and weighting techniques can be usedin such re-sizing, and known image processing technology can be used tomake other characteristics of the displayed images similar in a way thatfacilitates detection/diagnosis. When viewing such resized images,according to one aspect of the invention the merged image I_(MERGE) isautomatically resized accordingly.

The system described as a non-limiting example in this patentspecification is thus capable of receiving and displaying selectivelythe tomosynthesis projection images Tp, the tomosynthesis reconstructionimages Tr, the synthesized mammogram image Ms and/or the mammogramimages Mp, or a single type, or any sub combination of types. The systemhas software to perform reconstruction of tomosynthesis image data forimages Tp into images Tr, software for synthesizing mammogram images Msand software for merging a set of images to provide a supreme image thatdisplays, for every region of the merged image, the most relevantfeature in that region among all images in the image set.

For the purpose of this application, a feature is the ‘most relevant’based upon the application of one or more a computer assisted detection(CAD) algorithms to the image, wherein the CAD algorithms assignnumerical values, weights or thresholds, to pixels or regions based upondetected features within the region or between features. The featuresmay include, for example, speculated lesions, calcifications and thelike. Various systems and methods are currently known for computerizeddetection of abnormalities in radiographic images, such as thosedisclosed by Giger et al. in RadioGraphics, May 1993, pp. 647 656; Gigeret al. in Proceedings of SPIE, Vol. 1445 (1991), pp. 101 103; U.S. Pat.No. 4,907,156 to Doi et al.; U.S. Pat. No. 5,133,020 to Giger et al.;U.S. Pat. No. 5,343,390 to Doi et al.; U.S. Pat. No. 5,491,627 to Zhanget al.

FIG. 2 is a diagram which pictorially illustrates the merging of imagedata from a tomosynthesis image data set Tr, comprising tomosynthesisslices 10A to 10N, with image data from a mammogram 20, in this case asynthesized mammogram Ms. For ease of description filters are not shownin this example. The image data is forwarded to the region compare andimage merge processor 6 which compares the images on a region by regionbasis, searching for that image with the most desirable display data forthat region. The image with the most desirable display data may be thatimage with a highest pixel value, the lowest pixel value, or which hasbeen assigned a threshold value or weight based on the application of aCAD algorithm to the image. When the image with the most desirabledisplay data for that region is identified, the pixels of that regionare copied over to the corresponding region of the merged image 30. Forexample, as shown in FIG. 2 , region 36M from image Ms is written toregion 36I of the merged image 30, and region 35Tr of tomosynthesisslice 10A is copied to region 35I of the merged image 30.

As the regions of the merged image are populated, a merge map 40 isconstructed. The merge map 40 stores, for each region of the mergedimage 30, an identifier of the image which sourced the region.Therefore, as shown in FIG. 2 , the Ms identifier is stored in region36I, while the 10A TR slice identifier is stored in region 35I. As willbe described in more detail later herein, the merged map 40 may be usedduring merge image 30 display to permit fast viewing of the source imagefor a selected region.

Although the regions of FIG. 2 have been shown as pre-defined gridregions it is not necessary that regions be pre-defined in this manner.Rather, according to one aspect of the invention, the boundaries of theregions are dynamically identified during the region compare and imagegeneration process by performing comparisons at pixel or multi-pixelgranularities. FIG. 3 illustrates a merged image 50 which has beenconstructed via the combinations of numerous regions of different sourceimages, at arbitrary region boundaries 52. For example, as shown in FIG.3 , the boundaries 52 may be identified according to the detection ofparticular features within the slices.

FIG. 4 is a flow diagram 60 is provided to illustrate exemplary stepsthat may be performed in an image merge process of the presentinvention. At step 62 an image data set is acquired. The image data setmay be acquired by a tomosynthesis acquisition system, a combotomosynthesis/mammography system, or acquired merely by retrievingpre-existing data from a storage device, either locally or remotelylocated relative to an image display device. At step 64 a user mayoptionally select a merge mode. As part of selecting a merge mode, theuser may select which images to use for the merge, whether to highlightcertain features, such as calcifications, speculated lesions or masses,whether to display the image as a lower resolution tomosynthesis image,etc. At step 66 the images that are to be merged to build the supremeview are mapped to a common coordinate system, for example as describedin U.S. Pat. No. 7,702,142 entitled Matching Geometry Generation andDisplay of Tomosynthesis Images, incorporated herein by reference. Othermethods of matching images of different coordinate systems mayalternatively be used.

At step 68, image filters are applied and, for each of the regions(indicated by “step” 70), the process of comparing regions among thedifferent images begins, indicated by step 72. At step 74, eachI_(MERGE) region is populated with the pixels of the region of the imagein the image set having the most desirable pixels, value or pattern. Theprocess of populating regions continues until it is determined, at step76, that all regions have been evaluated, at which point the mergedimage is ready for display.

FIGS. 5A and 5B illustrate two views of a display 80. The first view ofdisplay 80 shown in FIG. 5A illustrates a merged image 82, havingregions sourced by different ones of an acquired or synthesized imageset. FIG. 5B illustrates one feature enabled by the present invention,whereby a user may select a region or area 83 within the merged image82, and the resulting image source 84 for that area is displayedtogether with the merged image. Of course the image need not bedisplayed proximate to the merged image; in one embodiment, selection ofa desired region replaces the merged image with the source image, oralternatively overlays the source image on the merged image, allowingthe two to be viewed in cine mode or toggle mode. Multiple source imagesmay be displayed concurrently in this manner, i.e., when a user selectsmultiple regions within the merged image a ‘stack’, pull down menu orother means is used for concurrent display of the multiple images. Themerged image may automatically be shrunk to an icon, or moved to a taskbar.

The merged image may also be dynamically modified by the selection ofdifferent filters, modes or sources at a user interface of the display.

According to another aspect of the invention, it is realized that amerged image may obfuscate data presented to the reviewer by essentiallyremoving depth information provided by the tomosynthesis reconstruction.For example, as shown in FIG. 6 , merged image 100 appears to includemultiple clusters of calcifications 102 a and 102 b, which are potentialpredictors of cancer. However, in a merged image, the respectiveclusters of calcifications 102 a and 102 b, which appear proximate whenevaluating only in the x-y axis, may actually be spread apart throughoutthe breast. To reduce the chance of a false positive cancer indication,the present invention includes a histogram 104, 106, which pictoriallyillustrates the depth of the clusters of calcifications 102 a and 102 bwithin the respective magnified regions. Histogram 104 illustrates thatthe cluster of calcifications 102 a are actually a real calcificationcluster, while Histogram 106 illustrates that the visualized cluster ofcalcifications 102 b are actually distributed throughout the breast.

FIG. 7 is schematic illustration of an x-ray image acquisition,processing and display system 108. The system 108 includes an x-raysource 110 that projects x-rays 112 at a detector 116 in order toacquire x-ray images of breast tissue under the control of anacquisition control 118. The acquired images are transmitted to aprocessing unit 120, which generates a reconstructed 3D image set 122, a2D synthesized image 124, and a synthesized merged image 126, asdescribed herein. These constructed/generated images may be stored in astorage memory 130 coupled to the processor 120 and/or displayed on aviewing interface 152 of a display unit 150.

Accordingly, a system and method for merging the most relevant data froma plurality of data sources to provide a ‘merged’ image I_(MERGE) fordisplay has been shown and described. The merged image may combineregions of any combination of 2D and 3D image data, including anacquired mammogram, a synthesized mammogram, or a tomosynthesisprojection image and a reconstructed 3D data set, thereby allowing theradiologist to quickly view a large number of regions of interest withina breast while referencing only a single 2D image and increasing theperformance and efficiency of breast cancer screening and diagnosis.

Having described exemplary embodiments, it can be appreciated that theexamples described above are only illustrative and that other examplesalso are encompassed within the scope of the appended claims. Forexample, flow diagrams are illustrative of exemplary steps; the overallimage merge may be achieved in a variety of manners using data mergemethods known in the art. The system block diagrams are similarlyrepresentative only, illustrating functional delineations that are notto be viewed as limiting requirements of the invention. Thus the abovespecific embodiments are illustrative, and many variations can beintroduced on these embodiments without departing from the spirit of thedisclosure or from the scope of the appended claims. For example,elements and/or features of different illustrative embodiments may becombined with each other and/or substituted for each other within thescope of this disclosure and appended claims.

1. A method for processing breast image data, the method comprising:obtaining a plurality of projection images Tp of a breast, each of theprojection images Tp in the plurality having a corresponding region;identifying features within the respective corresponding region of eachof the projection images Tp; assigning a weight to the respectivecorresponding region of each projection images Tp based on theidentified features; generating a synthesized 2D reconstructed imageslab Tr of the breast, the synthesized 2D reconstructed image slab Tralso having the corresponding region, wherein the synthesized 2Dreconstructed image slab Tr is generated by selecting a source imagefrom the plurality of projection images Tp to source image data for thecorresponding region in the synthesized 2D reconstructed image slab Trby comparing the assigned weights of the respective corresponding regionof the plurality of projection images Tp; generating, from theprojection images Tp, a synthesized 2D mammography image of an entirethickness of the breast, wherein the synthesized 2D mammography image ofthe entire thickness of the breast contains the corresponding region;displaying the synthesized 2D mammography image of the entire thicknessof the breast; receiving, in the displayed synthesized 2D mammographyimage of the entire thickness of the breast, a selection of thecorresponding region; and displaying the synthesized 2D reconstructedimage slab Tr based on the selection.
 2. The method of claim 1, furthercomprising generating a map for the synthesized 2D reconstructed imageslab Tr, wherein the map identifies the source image of thecorresponding region in the synthesized 2D reconstructed image slab Tr.3. The method of claim 1, wherein the identified features include atleast one of speculated lesions, masses, and calcifications.
 4. Themethod of claim 1, wherein the source image for the corresponding regionin the synthesized 2D reconstructed image slab Tr is selected based onhaving a comparatively highest assigned weight for the correspondingregion of all images of the plurality of projection images Tp.
 5. Themethod of claim 1, wherein the corresponding region in the synthesized2D reconstructed image slab Tr includes at least one pixel from thesource image.
 6. The method of claim 1, wherein a size of thecorresponding region in the synthesized 2D reconstructed image slab Tris statically pre-defined.
 7. The method of claim 1, wherein a size ofthe corresponding region in the synthesized 2D reconstructed image slabTr varies in accordance with a varying threshold of the source image. 8.A workstation for reviewing breast image data, the workstationcomprising: a processor; a display operatively coupled with theprocessor; and an interface operatively coupled with the processor,wherein the respective processor and interface are configured toselectively cause the processor to: obtain a plurality of projectionimages Tp of a breast, each of the projection images Tp in the pluralityhaving a corresponding region; identify features within the respectivecorresponding region of each of the projection images Tp; assign aweight to the respective corresponding region of each projection imagesTp based on the identified features; generate a synthesized 2Dreconstructed image slab Tr of the breast, the synthesized 2Dreconstructed image slab Tr also having the corresponding region,wherein the synthesized 2D reconstructed image slab Tr is generated byselecting a source image from the plurality of projection images Tp tosource image data for the corresponding region in the synthesized 2Dreconstructed image slab Tr by comparing the assigned weights of therespective corresponding region of the plurality of projection imagesTp; generate, from the projection images Tp, a synthesized 2Dmammography image of an entire thickness of the breast, wherein thesynthesized 2D mammography image of the entire thickness of the breastcontains the corresponding region; display the synthesized 2Dmammography image of the entire thickness of the breast; receive, in thedisplayed synthesized 2D mammography image of the entire thickness ofthe breast, a selection of the corresponding region; and display thesynthesized 2D reconstructed image slab Tr based on the selection. 9.The method of claim 1, wherein the respective processor and interfaceare further configured to selectively cause the processor to generate amap for the synthesized 2D reconstructed image slab Tr, wherein the mapidentifies the source image of the corresponding region in thesynthesized 2D reconstructed image slab Tr.
 10. The method of claim 1,wherein the identified features include at least one of speculatedlesions, masses, and calcifications.
 11. The method of claim 1, whereinthe source image for the corresponding region in the synthesized 2Dreconstructed image slab Tr is selected based on having a comparativelyhighest assigned weight for the corresponding region of all images ofthe plurality of projection images Tp.
 12. The method of claim 1,wherein the corresponding region in the synthesized 2D reconstructedimage slab Tr includes at least one pixel from the source image.
 13. Themethod of claim 1, wherein a size of the corresponding region in thesynthesized 2D reconstructed image slab Tr is statically pre-defined.14. The method of claim 1, wherein a size of the corresponding region inthe synthesized 2D reconstructed image slab Tr varies in accordance witha varying threshold of the source image.